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Topics 

• Diameters and their relationship to A2 

• Expanders 

• Butterfly networks 

1 Diameters and Eigenvalues 

So far, every time we've dealt with eigenvalues, it's had something to do with connectivity. For example, 
the spectral gap can be used to approximate the quality of cuts; it also describes how well a graph can mix 
under a random walk. They both are saying similar things: the eigenvalue is saying how connected a graph 
is. A walk will mix quickly if there's a lot connected to everything else. The min-cut will likewise be large 
if there's a lot of connectivity. 

For almost every reasonable property about a graph, there's something you can write down regarding its 
relation to the second eigenvalue of the Laplacian. Today, we're going to show the relation between A2 and 
the diameter of a graph. 

Definition 1 The diameter, 5, is the longest, shortest path between any two vertices of a graph. In other 
words, we can define the distance between two vertices u and v in G as the shortest path connecting the two. 
The diameter of G is the largest distance between any two vertices in G. 

It's not immediately clear why the diameter should be related to A2; the following provides intuition^: 

1. Well-connected graphs have big A2 

2. Well-connected graphs have a small 5 

3. So, graphs with big A2 should have small 5 

Before proceeding, we'll be making the following assumption: 
Assumption 2 G is a d-regular graph (this is for simplicity and not really a limiting assumption). 

We'll also be utilizing lazy random walks in our investigation. As a reminder. 
Definition 3 A lazy random walk is simply a random walk along a graph with self loops added in: 

M= - + - (1) 

Random Walk Self-loops 

where A is the graph's adjacency matrix and I is the identity matrix. 

Since we're using adjacency matrices, the interesting eigenvalues will be close to 1. So, let IJ.2 be the 
second largest eigenvalue of M and A be the gap (i.e., A = 1 — /U2). 



^Note, however, that this is not a proper syllogism 
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1.1 A First Bound 



Claim 4 For any G, 

(2) 

where S is the diameter of G, n is the number of vertices in the graph, and A = 1 — ^2, where fi2 is the 
second largest eigenvalue of the M associated with G. 

Claim 4 means that A, up to a logarithmic factor, really does provide a direct bound on the diameter of 
the graph. For most graphs, this isn't very tight, but it's a good place to start. So, why is it true? 
Proof We will use random walks to prove claim 4. Let u and v be vertices that are as far as possible from 
one another. Start a random walk at u and let pt{v) be the probability of a walk being at vertex v at time 
t. U pt{v) > 0, then 6 <t. Intuitively, this means that if we start at u and, after t time steps, there is some 
probability of ending up at v (the farthest vertex from «), then there must be a path of length t between 
the two. If there wasn't, pt{v) would be 0. 

Recalling that the stationary distribution is n{v) = 1/n for regular graphs, we can equivalently state that 
if \pt{v) — 'jt{v)\ < ^, then 5 < t. Why? Because if \pt{v) — 'k{v)\ < ^, then pt{v) > 0, implying 5 <t. 

Recall from our earlier lecture on random walks that 

iM^)-.(.)i<(i-A)y^^ = (i-Ar 

Since G is regular, d{v) = d for all vertices (allowing the last equality above). We'll now look at what 
happens when we set t = : 



(1-A)* = (l-A)"^ < 



-j^ \ In n ^ 



With the inequality coming from the fact that (1 — A)^/'^ < ^ for all A > 0. Thus, for t = we have that 

Inn 
X 



\pt{v) - Tr{v)\ < ^, and therefore, 6<t=^-^ 



1.2 A better bound 

As stated earlier, bounding 5 by ^ is not that great. So, can we do better? Yes. And we do so by using 
an important trick that frcxpiently comes up. First, note that if the ((/, l')''^ entry of A'' is non-zero, then 
there's a path of at most length k from u to v. Replacing A with M doesn't change this (it just makes the 
non-zero entries smaller). If e„ and e„ are basis vectors, then 

\pk{v) - tt{v)\ = \e^M^eu - - | < - 

n n 

which would imply 5 < k. Let's then let p{x) be a polynomial of degree k: 

k 

Note that we can also interpret M as a variable and apply p to it as follows: 

k 

p{M) = J2cjM^ 
j=i 

li p{M) has no zero entries, then S < k. Why? Because all non-zero elements of M indicate all vertices 
that can be reached in one step, is all vertices that can be reached in two steps, and so on. Thus, 
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for any non-zero entry (u.v) in p{M), there must have been a non-zero element in AP for < j < fc, 
implying the existence of a path from u to z; of at most length k. If this is true for all entries in p(M), then 
a path of at most length k exists from any vertex to any other vertex, which means the diameter is at most fc.^ 



Claim 5 Suppose p has degree k, p{l) = 1, and < ^ for all i>2, then 

5<k 



(3) 



Proof For this proof, it is sufficient to show that every entry in p{M) is non-zero. First, recall that we 
can write down any matrix, M, as the following: 

i 

where /U, and Vi are the i-th eigenvalue and eigenvector, respectively. Since 

i 

we can write 

k k ( ^ \ 

i=0 i=0 i i \j=0 j i 

Therefore, we can write the (a, 6)*^ entry of p(M) as follows 



elp{M)e^ 



> 



> 



> 



^P{l^i){e^Vi){vJ eb) 

i 

'^PiHi)viia)Vi{b) 

i 

1 " 

n 

'Y^p{li'i)'vAa)vi{b) 

1=1 

n 

^\p{l^i)\\vi{a)\\vi{h)\ 

i=2 

n 

max|p(/ii)|^|ui(o)| \vi{h)\ 



> 
> 



max \p{ij,i 

n i>2 



Where the penultimate step follows from J^i ^ 1- Let V be the matrix where rows are the 

eigenvectors Wj's. Then V ■ = / by the orthonormal condition. It follows that -V = I and the columns 



^Note that we're not saying anything about S if p(M) has zero entries. Since there are no restrictions on Cj, it's possible 
that the summation produces a zero entry for p{M) where for all positive Cj a non-zero entry would have existed. 



of V form an orthonormal basis. Hence (X]r=2l''^i(^)ll^*(^)l)^ — (Sr=2l'^j('^)l^) {^7=2\^i(^)\'^) — ^- "^^^ 
ultimate step follows from our assumption that < ^ for all i > 2. Thus, if p{l) = 1 and < ^ 

for all z > 2, we have that every entry in p{M) is non-zero, implying that S < k. M 

Claim 6 For any t e (0, 1), / assert the existence of a magic polynomial, p'f' , with the following properties: 

1. pf' is of degree k 

2. pt\l)^l 



3. 



pf{x) <2(l + \/2i) ^ for any x G [0, 1 — i] 



We will provide no proof for this claim here, but the polynomials are derived from Chebyshev polynomials, 
and we'll use them again later. To provide some intuition, figure 1 shows graphs of these polynomials for 
k = IQ with varying t. Notice that to keep the same bound for smaller values of t, a larger k is required due 
to the "oscillations" that the polynomial must take on in order to achieve = 1 while keeping p{x) small 
for X e [0,1 -t]. 





(a) (b) (c) 

Figure 1: (a) t = 0.1 (b) t = 0.001 (c) t = 0.001 zoomed in for x from 0.99 to 1 

If we set t = A, we get a degree k polynomial, p, such that 

1. p{l)^l 

2. \p{x)\ <2{l + V2A^ for any x G [0,^*2]- 



Additionally, if we set fc = |^1 + ln(2n), then it is possible to show that p{x) < ^ for all x G [0,^x2], 
which gives the following bound: 



<5 < 1 



In {2n) 



(4) 



This is much better than our previous bound of S < . So, strangely, by putting in a particular 
polynomial, we get a bound that grows with 1/y/X as opposed to just 1/A. This foreshadows our next unit 
on iterative linear algebra. 

1.3 Example Application 

Suppose that you have a symmetric matrix AI and want the eigenvector associated with the largest eigenvalue. 
For purposes of this example, let them be normalized such that the largest eigenvalue is 1. Then an easy 
way to get an approximate answer for the eigenvector is to compute M^x for a large k and a random x. 
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Why docs this give the eigenvector associated with //i = 1? If all other eigenvalues are less than 1. then for 
a large enough k, they will diminish in importance, until all that is left is vi. This is a very intuitive and 
natural algorithm that takes about 1 /A steps to get close. 

But we just found a much faster algorithm! Assuming that we know some good bound on A (if we don't, 

we could easily search for it), we can compute p'j^\M)x instead of M''x to get the dominant eigenvector. 
This method converges much faster, and we'll get into this more in a few lectures. 

2 Expanders 

If you had to know one set of graphs in your life, these are the ones to know. They often are a counterexample 
to many long-standing conjectures. Also, they turn up literally everywhere. If you didn't know any better, 
you would think that they don't exist from the described properties. But they're almost every single graph. 
Specifically, we'll be looking at families of d-regular graphs (G„)„ as n goes to infinity: 

Definition 7 (Gn)n is an expander family if X^iGn) > c for some constant c and for all n. 

Most of the graphs we've looked at are not expanders. For example, path graphs have A2 < 0(l/n^) 
and binary trees have A2 < 0(l/n). This means that A2 very quickly goes to zero as n ^ cxd for both cases. 
Expanders don't have this property. Even as n ^ 00, A2 stays above a constant. Given this, it's not clear 
that they should exist. 

Note: We should think of d as a constant. In other words, we'll pick a d and study expanders in that family. 
2.1 Relating Expanders to Cuts 

The first thing we'll look at is Cheeger's inequality for expanders. Recall that 

y < m 

For expanders, this implies 

I < m 

What does this mean? It means that any set, S, of vertices with l^l < n/2 has at least {c/2)\S\ edges 

leaving it. This is a strong property: for expanders, there are no small cuts that can be made in the graph. 
Every cut that balances the sizes of the sets of vertices cuts a constant fraction of the edges in the graph. 
The other side of Cheeger's inequality says 

e(i)«<A. 

Again, for expanders, this can be rewritten. 

Since d is a constant, this says that the isoperimetry, (j), is also bounded above by a constant. Nor- 
mally, there's a large gap between the upper-bound and lower-bounds in Cheeger's inequality. Here we've 
sandwiched ^ between two constants. Therefore, an equivalent definition of expanders is as follows: 

Definition 8 (G„)„ is expander family if (j){G) > c' for some constant d and all n. 
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2.2 Do Expanders Exist? 



A natural question is if expanders exist, what are the required parameters of the graphs? It turns out that 
random graphs are expanders, and so, almost all graphs are expanders. But, there's a limit as to how "good" 
of an expander you can have: 

Claim 9 For any G, 

A2 < d-2Vd - 1 + 0(1) (5) 

In other words, even though A2 is always larger than a constant, there's a limit as to how well-connected 
it can be. We won't prove this. But this is not that strong a bound. We already know that A2 < d. This is 
just 0{Vd) smaller. Furthermore, this bound is tight since Ramanujan graphs meet it. So that's the limit 
of what an expander can be. 



2.3 Expanders and Randomness 

Expanders are all over the study of randomness, but we'll just study one interesting property. We'll use 
/i2 = — A2 to simpHfy formulas, where now, 112 is the second largest eigenvalue of the adjacency matrix. 
Suppose you make a graph by randomly including each edge with probability d/n. In other words, construct 
a graph such that each vertex has an expected number of d, edges leaving it. Since the total number of 
possible edges is \S\\T\ and there's a. d/n probability of having each edge, the expected number of edges 
between any two sets S and T will be '^^'^^^''"^ . 



Claim 10 Expander Mixing Lemma: If you choose any two vertex sets, S and T, the difference in the total 
number of edges between the two and the expected number for a random graph is bounded. Formally, 



e{S,T) 



d\S\\T\ 



n 



< 



< 



n 



S\\T\\T\ 



M2Jniin(|5|,^)-min(|T|, T) 



This is surprising because there is no randomness here. This is just a property associated with expanders, 

but it behaves similarly to random graphs. 

Proof Let a and /3 be the fraction of total vertices that are in the sets S and T: 

\S\ = an \T\ = f3n. 
Let X and y be the characteristic vectors of S and T, respectively. 

Definition 11 A characteristic vector, x, of a set S is a vector of length n that has Xi = 1 if Vi G S, and 
Xi =0 otherwise. 

We can now write the number of edges between sets S and T as e{S, T) = Ay. Now, as you've probably 
noticed, it's beneficial to use vectors that are perpendicular to the all-ones vector, 1. So, we'll rewrite x and 
y as 

V = X — al w = y — (31. 

Clearly, v ■ 1 = w ■ 1 = Q, implying orthogonality. Rewriting the number of edges between sets S and T, we 
get 

e{S, T) = x'^Ay 

= (v + alf A{w + (31) 

= v'^Aw + v^A(31 + al^Aw + al^ A(31. 
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Using the following identities, 

we get 



Al = dl VA = dV, 



e{S, T) = Aw + I3v' Al + aV Aw + a/31' Al 
= v'^Aw + (3v^dl + adl^w + aj3dl^l 

= Aw + aPdn, 

where we have used the orthogonality of v and w with 1 (i.e., v^l = 0) to cancel out the middle two terms. 
We now have the following bound: 

\e{S,T) - aj3dn\ = \v'^Aw\ 

< \v\\Aw\ 

< \v\lJ,2\w\ 



n 

/"2 



^ Vian) ((1 - a)n) (pn) ((1 - (3)n) 



|5||S'||T||r| 



where the third line follows from the fact that w is orthogonal to 1, and thus, fi2 is the largest eigenvalue 
that can affect w, and the fourth line follows from the fact that \v\ = y^na(l — a). To see this, note that 

\v\ = la; — all 



'1^1(1 - a)2 + |5|(-a)2 

= \/an(l — a)2 + (1 — a)na'^ 
= \/ an(l — a)(l — a + a) 
= \/ an{l — a), 



and the same steps show that \w\ = ^n(3{l — (3). Thus, we have shown that |e(S', T) — al3dn\ < ^ ^\S\\S\\T\ |t| 

2.4 Some Properties We Now Know 

• Random walks on expanders mix in a logarithmic number of steps 

• Expanders have logarithmic diameter 

• Expanders have a constant isoperimetric number 

2.5 Vertex Expansion 

We've discussed cutting a graph and looking at the number of edges cut. An equivalent way of thinking 
about a cut is to select a set of vertices and then count the number of edges with one vertex in the set and 
one out. Another useful metric can be obtained by counting the number of vertices that are neighbors of a 
set. In other words, for a set of vertices, X, let N{X), be the set of vertices, Y, such that {x,y) G E such 
that X & X and y & X. 



Claim 12 

NiX)> '^^l^l 



fx-' + {d-' ~ IJ.^)\X\/n 
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We don't prove this, but the high level idea is the following: 

• Select a set of vertices from G. Call this set X. 

• Let Y be the set of vertices that are neither in X nor in N{X). In other words, Y = V\{N{X) U X). 

• Now, by construction, we have that e{X, Y) = 0. 

Algebra gets a little messy, but you can just plug the above into the expander mixing lemma to show 
this bound. It turns out also that for X/n small and n = 2\/d— 1, we can achieve 

7V(X)>^|X|. 

Why is this interesting? What this is saying is that for any set X. there are at least fi/4 neighbors not in X. 
Since each vertex has d neighbors total, this bound is quite strong. It turns out that this is about as good 
as you can get with spectral graph theory. To see this, we will generalize the vertex expansion as follows. 

We want to show bounds of the form |A^(S')| > 715*1. In other words, we want to say that the vertex 
expansion of G is greater than or equal to 7 for any S. Sometimes we'll only care about expansions of smaller 
sets (e.g., for l^] < O.Oln). 

Definition 13 G is an {a, (3) -expander if for a(3 < 1 and all sets S with \S\ < an have \N{S)\ > /3\S\. 

We showed that Ramanujan graphs are (a, (i/4) expanders for some constant a. Some applications need 
expansion greater than d/2 but with small (constant) a. These exist, but we can't prove better than d/2 
with spectral techniques.^ 

2.6 Bipartite Expanders 

Many of the applications of expanders use bipartite expanders. These are just expanders that are bipartite 
graphs. It is easier to show that these exists (it will be a homework problem!). 

Definition 14 A d-regular bipartite graph is an {a, P)- expander if every set S on the left with \S\ < an has 
N{S) > 0\S\. 

Whenever a/3 < 1, there exists some d such that almost all d-regular graphs on n nodes (for n sufficiently 
large), are (a, /3)-expanders. 



^It turns out that random graphs work here. In 2002, Capalbo, Reingold, Vadhan, and Wigderson gave an expUcit construc- 
tion technique with expansion d — o(l). 
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